Systems and methods for inserting contextual advertisements into a virtual environment

ABSTRACT

Systems and methods are described herein for inserting contextual advertisements into a virtual environment. While presenting a virtual environment, a media device receives speech input. The media device selects an advertisement from an advertisement database based on the speech. The media device then determines the current environment of the virtual environment and generates a virtual object that is related to the current environment. The media device overlays the selected advertisement over the generated virtual object and displays the generated virtual object in the virtual environment.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.17/218,846, filed Mar. 31, 2021, which is a continuation of U.S. patentapplication Ser. No. 16/226,955, filed Dec. 20, 2018, now U.S. Pat. No.10,997,630. The disclosures of which are hereby incorporated byreference herein in their entireties.

BACKGROUND

The present disclosure relates to content delivery and consumptionsystems and, more particularly, advertisement insertion in virtualenvironments.

SUMMARY

Advertisement insertion in media content takes several forms.Preselected advertisements or targeted advertisements may be insertedinto traditional forms of media content at predefined insertion pointsor triggered by user input. Virtual environments, however, do not easilylend themselves to traditional advertisement insertion techniques. Auser may interact with a virtual reality video game in which the usermay freely move about the game environment. Sudden and intrusiveinsertion of advertisements disrupts the user's experience. A user maybe participating in a virtual meeting and cannot be distracted bytraditional advertisements. Thus, a method of inserting advertisementsinto virtual environments in a natural, unobtrusive way is required.

Systems and methods are described herein for generating a virtual objectfor insertion into a virtual environment on which a contextualadvertisement may be placed. Virtual objects may be introduced into avirtual environment in a natural way. For example, if the environmentdisplayed in the virtual environment is a city street, a new vehicle canbe inserted as driving into view on the street. Such object insertion isunobtrusive and does not disrupt the user's experience. A contextualadvertisement can then be displayed on the vehicle so that the userviews the advertisement as the vehicle drives through the user's fieldof view.

While presenting a virtual environment, a media device receives speechinput. The media device selects an advertisement from an advertisementdatabase based on the speech. The media device then determines thecurrent environment of the virtual environment and generates a virtualobject that is related to the current environment. The media deviceoverlays the selected advertisement over the generated virtual objectand displays the generated virtual object.

The media device may identify a plurality of virtual objects currentlybeing displayed in the virtual environment and retrieve metadata of oneof the displayed virtual objects. The media device determines whetherthe metadata of the virtual object uniquely identifies a singleenvironment. If so, the media device identifies the single environmentas the current environment. If not, the media device retrieves metadatafor additional virtual objects until a unique environment is identified.The media device may identify a unique environment from a single virtualobject or may compare sets of environments identified by a number ofvirtual objects to find a single environment common to all the sets ofenvironments.

The media device may determine whether a virtual object currentlydisplayed in the virtual environment is suitable for displaying theselected advertisement and generate a new virtual object only if nosuitable virtual object is currently displayed. To determine if asuitable virtual object is currently being displayed, the media deviceidentifies a plurality of virtual objects currently being displayed inthe virtual environment, and further identifies a respective type ofeach of the virtual objects. The media device then determines whetherany of the virtual objects is of a type that is suitable for displayingthe selected advertisement.

To display the selected advertisement on a virtual object, the mediadevice identifies a surface of the virtual object suitable fordisplaying the selected advertisement and displays the selectedadvertisement on the identified surface. The media device may determinean orientation of the virtual object and transform an image of theselected advertisement to match the orientation of the virtual object.

To select an advertisement, the media device identifies a category ofadvertisement related to the speech and determines a particularadvertisement based on a user profile and the identified category. Themedia device then selects that particular advertisement from theadvertisement database for display.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects and advantages of the disclosure will beapparent upon consideration of the following detailed description, takenin conjunction with the accompanying drawings, in which like referencecharacters refer to like parts throughout, and in which:

FIG. 1 shows a generalized embodiment of insertion of a contextualadvertisement in a virtual environment in accordance with someembodiments of the disclosure;

FIG. 2 shows a second generalized embodiment of insertion of acontextual advertisement in a virtual environment in accordance withsome embodiments of the disclosure;

FIG. 3 shows a third generalized embodiment of insertion of a contextualadvertisement in a virtual environment in accordance with someembodiments of the disclosure;

FIG. 4 shows a fourth generalized embodiment of insertion of acontextual advertisement in a virtual environment in accordance withsome embodiments of the disclosure;

FIG. 5 is a data flow diagram for insertion of a contextualadvertisement in a virtual environment in accordance with someembodiments of the disclosure;

FIG. 6 is a diagram showing selection of an advertisement for insertionbased on user preferences in accordance with some embodiments of thedisclosure;

FIG. 7 is a block diagram representing control circuitry and data flowwithin a media device for insertion of a contextual advertisement in avirtual environment in accordance with some embodiments of thedisclosure;

FIG. 8 is a flowchart representing a process for inserting a contextualadvertisement in a virtual environment in accordance with someembodiments of the disclosure;

FIG. 9 is a flowchart representing a process for identifying a currentenvironment in accordance with some embodiments of the disclosure;

FIG. 10 is a flowchart representing a second process for identifying acurrent environment in accordance with some embodiments of thedisclosure;

FIG. 11 is a flowchart representing a process for determining whether togenerate a new virtual object for display of a contextual advertisementin accordance with some embodiments of the disclosure;

FIG. 12 is a flowchart representing a process for displaying acontextual advertisement on a virtual object in accordance with someembodiments of the disclosure;

FIG. 13 is a flowchart representing a process for placing a contextualadvertisement on a virtual object in accordance with some embodiments ofthe disclosure;

FIG. 14 is a flowchart representing a process for selecting a contextualadvertisement for insertion in accordance with some embodiments of thedisclosure; and

FIG. 15 is a flowchart representing a process for selecting a virtualobject on which to display a contextual advertisement in accordance withsome embodiments of the disclosure.

DETAILED DESCRIPTION

FIG. 1 depicts a virtual environment 100 as viewed from a user'sperspective on a media device. Virtual environment 100 may be a videogame in which the user is participating with several other users. Theusers may communicate 102 with each other through the video game, eithervocally or with text messages. During gameplay, a user may ask otherusers if, for example, they should break from the video game to orderpizza. The media device monitors the communications and detects akeyword 104 “pizza.” In response to detecting the keyword, the mediadevice retrieves an advertisement for a product matching the keywordfrom an advertisement database. The specific advertisement retrieved maybe influenced by the preference of a particular user. For example, inresponse to the keyword “pizza,” a first user's media device mayretrieve an advertisement for “Joe's Pizza” while a second user's mediadevice may retrieve an advertisement for “Bob's Pizza.” The virtualenvironment 106, presented on the first user's media device, is modifiedby the first media device to include a new virtual object on which isdisplayed the “Joe's Pizza” advertisement 110. Likewise, the virtualenvironment 108, presented on the second user's media device, ismodified by the second media device to include a new virtual object onwhich is displayed the “Bob's Pizza” advertisement 112. It should benoted that while FIG. 1 depicts insertion of the same virtual object inboth virtual environments 106 and 108, each media device may select anyappropriate virtual object on which to display the selectedadvertisement, as described below.

FIG. 2 depicts a virtual environment 200, which is a virtual meeting.During the meeting, a participant may, for example, suggest 202 breakingfor coffee. As above, the media device through which the meeting isbeing presented monitors the participants' speech for keywords anddetects the keyword 204 “coffee.” In response to detecting the keyword,the media device retrieves a coffee advertisement from an advertisementdatabase and generates a new virtual object on which to display theadvertisement. The virtual object 208 is then displayed.

FIG. 3 depicts an augmented reality display on a media device 300. Auser may enter a search 302 on media device 300 for nearby restaurants.Media device 300 may access an advertisement database and retrieve anadvertisement for a local restaurant. Media device 300 may use userpreferences to determine a type of restaurant for which an advertisementshould be retrieved. For example, media device 300 may determine thatthe user likes pizza and retrieves an advertisement for “Joe's Pizza”from the advertisement database. Media device 300 captures an image ofthe surrounding environment using a built-in camera and determines anappropriate virtual object to overlay over the image. For example, mediadevice 300 determines that the user is on a city street. Based on thedetermined environment, media device 300 selects a delivery bicycle 304on which to display the retrieved advertisement 306. Media device 300overlays the bicycle with the advertisement over the captured image. Theposition of the virtual object may be static or animated so as to movealong the street. Animation of the virtual object may be linked tomovement of the media device 300 in order to keep the virtual objectdisplayed for a longer period of time.

FIG. 4 depicts a virtual environment 400 with which users cannotinteract, such as a cutscene in a video game, or broadcast videocontent. In response to detecting the keyword “pizza,” as above, themedia device on which the virtual environment is presented determinesthe currently depicted environment and selects a virtual object toinsert into the virtual environment on which to display anadvertisement. In the example of FIG. 4 , the environment is a race. Arunner is displayed running past a crowd of spectators. The media devicedetermines that a barrier 404 may be placed between the runner and thespectators and provides a suitable surface on which to display theselected advertisement 406.

FIG. 5 is a data flow diagram for insertion of a contextualadvertisement in a virtual environment. A media device, such as mediadevice 300, receives user speech at speech input 500. The media deviceidentifies and extracts from the speech a keyword and, at 502, uses thekeyword to retrieve an advertisement from advertisement database 504. At506, at advertisement related to the keyword is transferred from theadvertisement database 504 to an object insertion API 508. At 510, anidentifier of the current virtual environment is retrieved by the objectinsertion API 508 from the current environment 512. At 514, the objectinsertion API 508 selects a virtual object related to the currentenvironment using the identifier of the current environment and placesthe retrieved advertisement on the selected virtual object. At 516,object insertion API 508 inserts the virtual object into the displayedcontent 518.

FIG. 6 is a diagram showing selection of an advertisement for insertionbased on user preferences. Table 600 represents a set of userpreferences for various products. User preferences may be learned frommonitoring user behaviors, such as Internet shopping, Internet searchhistories, content consumption histories, and the like. User preferencesmay be stored on a media device or on a remote server accessible by amedia device. Stored user preferences may include product 602, specificbrands 604 for each product, and an expiration date or time 606 for eachproduct/brand association. For example, for the product “pizza” 608, auser may prefer “Joe's Pizza” 610, which may be learned from the user'sorder history using services such as Seamless®, GrubHub® or Uber Eats®.The media device may monitor the time of such orders and determine thatthe user never orders pizza after 10:00 PM. The media device thereforesets the expiration 612 for “Joe's Pizza” as 10:00 PM to prevent pizzaadvertisements from being presented to the user at times when the userwould not order pizza.

FIG. 7 is an illustrative block diagram representing circuitry and dataflow within a media device 700 in accordance with some embodiments ofthe disclosure. Media device 700 may be any device for accessing thecontent described above, such as a television, a Smart TV, a set-topbox, an integrated receiver decoder (IRD) for handling satellitetelevision, a digital storage device, a digital media receiver (DMR), adigital media adapter (DMA), a streaming media device, a DVD player, aDVD recorder, a connected DVD, a local media server, a BLU-RAY player, aBLU-RAY recorder, a personal computer (PC), a laptop computer, a tabletcomputer, a WebTV box, a personal computer television (PC/TV), a PCmedia server, a PC media center, a handheld computer, a stationarytelephone, a personal digital assistant (PDA), a mobile telephone, aportable video player, a portable music player, a portable gamingmachine, a smart phone, or any other television equipment, computingequipment, or wireless device, and/or combination of the same. Mediadevice 700 comprises input circuitry 702. Input circuitry 702 mayinclude a microphone and voice processing circuitry for receiving voicecommands. Media device 700 also comprises control circuitry 704 andstorage 706. Control circuitry 704 may be based on any suitableprocessing circuitry and comprises control circuits and memory circuitsthat may be disposed on a single integrated circuit or may be discretecomponents. As referred to herein, processing circuitry should beunderstood to mean circuitry based on one or more microprocessors,microcontrollers, digital signal processors, programmable logic devices,field-programmable gate arrays (FPGAs), application-specific integratedcircuits (ASICs), etc., and may include a multi-core processor (e.g.,dual-core, quad-core, hexa-core, or any suitable number of cores). Insome embodiments, processing circuitry may be distributed acrossmultiple separate processors or processing units, for example, multipleof the same type of processing units (e.g., two Intel Core i7processors) or multiple different processors (e.g., an Intel Core i5processor and an Intel Core i7 processor). Some control circuits may beimplemented in hardware, firmware, or software. Input circuitry 702 maybe integrated with control circuitry 704. Storage 706 may be any devicefor storing electronic data, such as random-access memory, read-onlymemory, hard drives, optical drives, digital video disc (DVD) recorders,compact disc (CD) recorders, BLU-RAY disc (BD) recorders, BLU-RAY 3Ddisc recorders, digital video recorders (DVR, sometimes called apersonal video recorder, or PVR), solid state devices, quantum storagedevices, gaming consoles, gaming media, or any other suitable fixed orremovable storage devices, and/or any combination of the same.

Control circuitry 704 comprises virtual content processing circuitry708. Virtual content processing circuitry 708 processes two- andthree-dimensional video content and associated audio content, which maybe received via a physical RF channel over a cable television connectionor terrestrial broadcast, or may be received over an Internet connectionfrom an over-the-top (OTT) service using a wired connection (e.g.,Ethernet) or wireless connection (e.g., 802.11a/b/g/n (WiFi), WiMax,GSM, UTMS, CDMA, TDMA, 3G, 4G, 4G LTE, or any other suitable type ofwireless data transmission). Virtual content processing circuitry 708transfers audio data 710 and video data 712 of the virtual content toaudio output circuitry 714 and video output circuitry 716, respectively.

During presentation of the virtual content, input circuitry 702 receivesuser speech. Input circuitry 702 transfers 718 the speech to controlcircuitry 704 where it is received by language processing circuitry 720.Language processing circuitry 720 processes the speech using any knownsuitable speech processing technique to identify individual words.Language processing circuitry 720 compares 722 the identified words toproduct-related keywords in user preferences 724 stored in storage 706.If a keyword is detected in the speech, control circuitry 704 accesses726 advertisements database 728 in storage 706. For example, languageprocessing circuitry 720 uses the detected keyword to query theadvertisements database 728 for an advertisement related to the detectedkeyword. The query may also contain a specific brand identified by userpreferences 724 as being associated with the keyword. An advertisementis then selected and returned 730 from the advertisements database 728to control circuitry 704, where it is received by image processingcircuitry 732.

Control circuitry 704, using virtual content processing circuitry 708,determines a current environment of the virtual content. For example,virtual content processing circuitry 708 may access metadata of thevirtual environment to identify a current environment (e.g., mountain,beach, city street, etc.). Alternatively, virtual content processingcircuitry 708 may identify a number of virtual objects currently beingdisplayed in the virtual environment. Virtual content processingcircuitry 708 may access metadata or other identifying information foreach displayed virtual object to determine an environment in which therespective virtual object belongs. For example, a stop sign may bedisplayed on a city street while a jellyfish may be displayed at abeach. Once the current environment has been identified, virtual contentprocessing circuitry 708 selects a virtual object on which to displaythe selected advertisement. In some cases, a currently displayed objectmay be suitable for displaying the selected advertisement. If nocurrently displayed object is suitable for displaying the selectedadvertisement, virtual content processing circuitry 708 may select asuitable object appropriate to the current environment from a library ofobjects. Virtual content processing circuitry transfers the selectedvirtual object, or geometric and topological parameters thereof, toimage processing circuitry 732. Image processing circuitry 732 processesthe selected advertisement for display on a surface of the selectedvirtual object. For example, image processing circuitry 732 maytransform the selected advertisement to match the tilt, skew, or viewingangle of a surface of the selected virtual object and place it on thesurface. The modified virtual object is then transferred 736 from imageprocessing circuitry 732 to virtual content processing circuitry 708 forinsertion to the virtual environment.

FIG. 8 is a flowchart representing an illustrative process for insertinga contextual advertisement in a virtual environment in accordance withsome embodiments of the disclosure. The process 800 represented in FIG.8 may be implemented on control circuitry 704, and one or more actionsof process 800 may be incorporated into or combined with one or moreactions of any other process or embodiment described herein.

At 802, control circuitry 704 receives, from input circuitry 702, audioinput representing speech. For example, input circuitry 702 may includea microphone for receiving audio input from users. Alternatively oradditionally, input circuitry 702 may include wireless receivercircuitry for receiving analog and/or digital audio signals via radio,Bluetooth, WiFi, or any other suitable transmission protocol. Inputcircuitry 702 transfers the audio signal to control circuitry 704 forprocessing. Control circuitry 704, using language processing circuitry720, processes the audio signal to identify individual words containedin the speech. Language processing circuitry 720 may use any suitablespeech processing technique.

At 804, control circuitry 704 compares the words contained in the speechto a list of keywords. Control circuitry 704 accesses a list of keywordsfrom storage 706. The keywords may be stored as part of user preferences724. For example, certain generic product types such as “car,” “game,”or “pizza” may be keywords. At 806, control circuitry 704 determineswhether the speech contains a keyword. If not, control circuitry 704continues to monitor received speech. If a keyword is contained in thespeech, then, at 808, control circuitry 704 determines whether thecontext of the keyword indicates that an advertisement should beinserted. For example, language processing circuitry 720 analyzes thesentence in which the keyword was detected. If the keyword was thesubject of a question, such as “Should we order pizza?”, languageprocessing circuitry 720 determines that an advertisement for pizzashould be inserted. However, if the keyword was said in passing, such as“I had pizza for dinner last night”, language processing circuitry 720determines that an advertisement for pizza should not be inserted.

If an advertisement should be inserted, then, at 810, control circuitry704 selects an advertisement from an advertisement database based on thekeyword. Control circuitry 704 may query the advertisement database forad advertisement related to the keyword “pizza.” Control circuitry 704may also access user preferences 724 to retrieve a specific brandassociated with the keyword. For example, user preferences may associate“Joe's Pizza” with the keyword “pizza.” Control circuitry 704 thereforequeries the advertisement database for a “Joe's Pizza” advertisement.

At 812, control circuitry 704 determines a current environment of thevirtual environment. Control circuitry 704, using virtual contentprocessing circuitry 708, determines a particular environment or setting(e.g., city street, beach, mountain) being displayed in the virtualenvironment. This may be accomplished by accessing metadata of thevirtual environment, or by using methods described below in connectionwith FIGS. 9 and 10 .

At 814, control circuitry 704 determines whether a currently displayedvirtual object is suitable for displaying the selected advertisement.This may be accomplished using methods described below in connectionwith FIG. 11 . If no currently displayed virtual object is suitable fordisplaying the selected advertisement then, at 816, control circuitry704 generates a virtual object that is related to the currentenvironment for display of the selected advertisement. This may beaccomplished using methods described below in connection with FIG. 15 .At 818, control circuitry 704 overlays the selected advertisement oneither the generated virtual object or the suitable currently displayedvirtual object. At 820, control circuitry 704 displays the virtualobject with the selected advertisement overlaid on it.

The actions or descriptions of FIG. 8 may be used with any otherembodiment of this disclosure. In addition, the actions and descriptionsdescribed in relation to FIG. 8 may be done in suitable alternativeorders or in parallel to further the purposes of this disclosure.

FIG. 9 is a flowchart representing an illustrative process foridentifying a current environment in accordance with some embodiments ofthe disclosure. The process 900 represented in FIG. 9 may be implementedon control circuitry 704, and one or more actions of process 900 may beincorporated into or combined with one or more actions of any otherprocess or embodiment described herein.

At 902, control circuitry 704 identifies the number of virtual objectscurrently being displayed and initializes a counter variable i with avalue of 0. To identify the number of virtual objects being displayed,control circuitry 704 may poll, query, or otherwise request from virtualcontent processing circuitry 708 an index, list, array, database, orother data structure containing at least one identifier of each virtualobject currently being processed by virtual content processing circuitry708 for display in the virtual environment. At 904, control circuitry704 retrieves metadata of i^(th) virtual object. Control circuitry 704may retrieve metadata from the virtual content directly or may retrievean identifier of the virtual object and query a virtual object librarydatabase using the identifier.

At 906, control circuitry 704 determines whether the metadata of thevirtual object identifies a single environment. For example, metadatafor a palm tree may identify a beach environment but may also identify acity street environment in a tropical climate. A surfer on a surfboard,however, might identify only a beach because no other environment wouldinclude such an object. If a single environment is identified, thenprocessing proceeds to 814 above. If more than one environment isidentified, then, at 808, control circuitry determines whether metadatafor all currently displayed virtual objects has been retrieved. If not,then, at 910, control circuitry increments the value of i, andprocessing returns to 904. If metadata for all currently displayedobjects has been retrieved, then processing return to 902, where controlcircuitry 704 again identifies a number of virtual objects currentlydisplayed.

The actions or descriptions of FIG. 9 may be used with any otherembodiment of this disclosure. In addition, the actions and descriptionsdescribed in relation to FIG. 9 may be done in suitable alternativeorders or in parallel to further the purposes of this disclosure.

FIG. 10 is a flowchart representing a second illustrative process foridentifying a current environment in accordance with some embodiments ofthe disclosure. The process 1000 represented in FIG. 10 may implementedon control circuitry 704, and one or more actions of process 1000 may beincorporated into or combined with one or more actions of any otherprocess or embodiment described herein.

At 1002, control circuitry 704 retrieves a first set of environmentsidentified by metadata of a first virtual object. At 1004, controlcircuitry 704 retrieves a second set of environments identified bymetadata of a second virtual object. These steps may be accomplished asdescribed above in connection with FIG. 9 . At 1006, control circuitry704 compares the two sets of environments and, at 1008, determineswhether there is a single common environment between the two sets ofenvironments. If there is no single common environment between the twosets of environments, then, at 1010, control circuitry 704 retrievesanother set of environments identified by metadata of another virtualobject. Processing then returns to 1006, where control circuitry 704compares all retrieved sets of environments to identify a single commonenvironment between them. If there is a single common environmentbetween the sets of environments, then, at 1012, control circuitry 704identifies the single common environment as the current environment ofthe virtual environment.

It is contemplated that the actions or descriptions of FIG. 10 may beused with any other embodiment of this disclosure. In addition, theactions and descriptions described in relation to FIG. 10 may be done inalternative orders or in parallel to further the purposes of thisdisclosure.

FIG. 11 is a flowchart representing an illustrative process fordetermining whether to generate a new virtual object for display of acontextual advertisement in accordance with some embodiments of thedisclosure. The process 1100 represented in FIG. 11 may be implementedon control circuitry 704, and one or more actions of process 1100 may beincorporated into or combined with one or more actions of any otherprocess or embodiment described herein.

At 1102, control circuitry 704 identifies a number of virtual objectscurrently being displayed. This may be accomplished using methodsdescribed above in connection with FIG. 9 .

At 1104, control circuitry 704 identifies a respective type of eachvirtual object currently being displayed. For example, control circuitry704 may identify from metadata associated with each respective virtualobject whether the respective virtual object is a foreground object or abackground object. Control circuitry 704 may alternatively oradditionally determine from the metadata whether the respective virtualobject is a static object (i.e., the position of the object does notchange relative to background objects) or an animated object (i.e., theposition of the object changes relative to background objectsindependently of a user's movement within the virtual environment).Control circuitry 704 may also identify whether the respective virtualobject represents a natural object or a man-made object.

At 1106, control circuitry 704 determines whether any object currentlybeing displayed is of a type suitable for displaying the selectedadvertisement. For example, in determining whether a virtual objectrepresents a natural object (e.g., a tree, cloud, grass, bird, face,etc.) or a man-made object (e.g., building, vehicle, billboard, T-shirt,etc.), control circuitry 704 may further determine whether the objectrepresented by a particular virtual object would normally include anadvertisement. If a virtual object currently being displayed is of atype suitable for displaying the selected advertisement, then, at 1108,control circuitry 704 selects the object of a suitable type for displayof the selected advertisement. If no virtual objects of a suitable typeare currently being displayed, then, at 1110, control circuitry 704generates a new virtual object related to the current environment asdescribed above in connection with FIG. 8 .

It is contemplated that the actions or descriptions of FIG. 11 may beused with any other embodiment of this disclosure. In addition, theactions and descriptions described in relation to FIG. 11 may be done inalternative orders or in parallel to further the purposes of thisdisclosure.

FIG. 12 is a flowchart representing an illustrative process fordisplaying a contextual advertisement on a virtual object in accordancewith some embodiments of the disclosure. The process 1200 represented inFIG. 12 may be implemented on control circuitry 704, and one or moreactions of process 1200 may be incorporated into or combined with one ormore actions of any other process or embodiment described herein.

At 1202, control circuitry 704 determines the number of surfaces of avirtual object and initializes a counter variable i with a value of 0.To determine the number of surfaces of a virtual object, controlcircuitry 704 may retrieve geometric and/or topological parameters ofthe virtual object. If the virtual object is a polygon, controlcircuitry 704 may count the faces of the polygon to determine the numberof surfaces. If the virtual object is an irregular object (e.g., anobject with curved surfaces), control circuitry 704 may use topologicaldata to determine the number of surfaces.

At 1204, control circuitry 704 determines whether the i^(th) surface ofthe virtual object is suitable for displaying the selectedadvertisement. For example, control circuitry 704 determines whether thei^(th) surface is currently facing the user, determines the size of thesurface, determines the curvature of the surface, and/or determines thelighting of the surface in order to identify whether the surface issuitable for displaying the selected advertisement. If so, then, at1210, control circuitry 704 displays the selected advertisement on thei^(th) surface. If not, then, at 1206, control circuitry 704 determineswhether all the surfaces of the virtual object have been considered. Ifnot, control circuitry 704, at 1208, increments the value of i andprocessing returns to 1204.

It is contemplated that the actions or descriptions of FIG. 12 may beused with any other embodiment of this disclosure. In addition, theactions and descriptions described in relation to FIG. 12 may be done inalternative orders or in parallel to further the purposes of thisdisclosure.

FIG. 13 is a flowchart representing an illustrative process for placinga contextual advertisement on a virtual object in accordance with someembodiments of the disclosure. The process 1300 represented in FIG. 13may be implemented on control circuitry 704, and one or more actions ofprocess 1300 may be incorporated into or combined with one or moreactions of any other process or embodiment described herein.

At 1302, control circuitry 704, using virtual content processingcircuitry 708, determines a currently displayed viewing angle relativeto a fixed reference position in the virtual environment. Virtualcontent processing circuitry 708 identifies a fixed reference positionfrom which all virtual objects are arrayed in a sphere around the user'sposition within the virtual environment. Any point in the virtualenvironment can thus be identified by a set of coordinates describing anumber of degrees from the reference point in the horizontal plane, anumber of degrees above or below the reference point in the verticalplane. Virtual content processing circuitry 708 identifies thecoordinates of the point in the center of the user's field of view andcalculates the angle between the point and the reference position.

At 1304, control circuitry 704, using virtual content processingcircuitry 708, determines an orientation of the virtual object relativeto the reference position. For example, a single point, surface edge, orvertex of the virtual object may be tracked by virtual contentprocessing circuitry 708 as a reference point from which to display thevirtual object. Virtual content processing circuitry 708 may compare thevirtual object reference point with the reference position to determinethe orientation of the virtual object.

At 1306, control circuitry 704, using virtual content processingcircuitry 708, calculates the currently displayed angle of the surfaceof the virtual object on which the selected advertisement is to bedisplayed. Virtual content processing circuitry 708 uses geometry ortrigonometry to calculate a position and viewing angle of the virtualobject relative to the center of the user's field of view. Then, at1308, control circuitry 704, using image processing circuitry 732,calculates an image transformation matrix that matches the currentlydisplayed angle of the surface of the virtual object. At 1310, imageprocessing circuitry 732 transforms an image of the selectedadvertisement using the transformation matrix so that the selectedadvertisement may seamlessly be displayed on the virtual object.

The above calculations and transformations can also be made using aprojected position at which newly generated virtual object may be placedwithin the virtual environment.

It is contemplated that the actions or descriptions of FIG. 13 may beused with any other embodiment of this disclosure. In addition, theactions and descriptions described in relation to FIG. 13 may be done inalternative orders or in parallel to further the purposes of thisdisclosure.

FIG. 14 is a flowchart representing an illustrative process forselecting a contextual advertisement for insertion in according withsome embodiments of the disclosure. The process 1400 represented in FIG.14 may be implemented on control circuitry 704, and one or more actionsof process 1400 may be incorporated into or combined with one or moreactions of any other process or embodiment herein.

At 1402, control circuitry 704 identifies a category of advertisementrelated to the speech. For example, the speech may include the keyword“slice” and, using language processing circuitry 720, control circuitry704 may genericize this keyword to the category of food. At 1404,control circuitry 704 determines whether a user profile associates aparticular advertisement with the category. Continuing the aboveexample, control circuitry may determine whether user preferences 724include a particular brand (e.g., “Joe's Pizza”) or other specificadvertisement identifier in the category of food. If the user profiledoes associate a particular advertisement with the category, then, at1406, control circuitry 704 retrieves an identifier of the particularadvertisement and, at 1408, selects the particular advertisement from anadvertisement database using the retrieved identifier. If noadvertisement is associated with the category, then, at 1410, controlcircuitry selects an advertisement related to the category from theadvertisement database at random.

It is contemplated that the actions or descriptions of FIG. 14 may beused with any other embodiment of this disclosure. In addition, theactions and descriptions described in relation to FIG. 14 may be done inalternative orders or in parallel to further the purposes of thisdisclosure.

FIG. 15 is a flowchart representing an illustrative process forselecting a virtual object on which to display a contextualadvertisement. The process 1500 represented in FIG. 15 may beimplemented on control circuitry 704, and one or more actions of process1500 may be incorporated into or combined with one or more actions ofany other process or embodiment described herein.

At 1502, control circuitry 704 identifies a number of virtual objectsrelated to the current environment. For example, control circuitry 704may access a virtual object library and filter the virtual objects tothose related to the current environment. At 1504, control circuitry 704identifies a type of each identified virtual object. This may beaccomplished using methods described above in connection with FIG. 11 .At 1506, control circuitry initializes a counter variable N with a valueof 0, and, at 1508, determines whether the N^(th) object is of a typesuitable for displaying the selected advertisement. This may beaccomplished using methods described above in connection with FIG. 11 .If the object is not of a suitable type, then, at 1510, controlcircuitry determines whether there are more virtual objects to consider.If so, then, at 1512, control circuitry increments the value of N andprocessing returns to 1508. If the N^(th) object is of a suitable typefor displaying the selected advertisement then, at 1514, controlcircuitry 704 selects the N^(th) virtual object to display the selectedadvertisement.

It is contemplated that the actions or descriptions of FIG. 15 may beused with any other embodiment of this disclosure. In addition, theactions and descriptions described in relation to FIG. 15 may be done inalternative orders or in parallel to further the purposes of thisdisclosure.

The processes described above are intended to be illustrative and notlimiting. One skilled in the art would appreciate that the steps of theprocesses discussed herein may be omitted, modified, combined, and/orrearranged, and any additional steps may be performed without departingfrom the scope of the invention. More generally, the above disclosure ismeant to be exemplary and not limiting. Only the claims that follow aremeant to set bounds as to what the present invention includes.Furthermore, it should be noted that the features and limitationsdescribed in any one embodiment may be applied to any other embodimentherein, and flowcharts or examples relating to one embodiment may becombined with any other embodiment in a suitable manner, done indifferent orders, or done in parallel. In addition, the systems andmethods described herein may be performed in real time. It should alsobe noted that the systems and/or methods described above may be appliedto, or used in accordance with, other systems and/or methods.

What is claimed is:
 1. A method comprising: receiving, duringpresentation of a virtual environment, a user input comprising a searchentry or an audio input; selecting an advertisement from anadvertisement database based on the user input; determining, by aprocessor, that each respective surface of a number of surfacesdisplayed in the virtual environment is not suitable for placement ofthe advertisement based on (a) lighting of the respective surface and(b) curvature of the respective surface, as viewed from a perspective ofa user from which the user input was received; and in response to thedetermining that each respective surface of the number of surfacesdisplayed in the virtual environment is not suitable for the placementof the advertisement: determining a current environment of the virtualenvironment; generating for display a virtual object related to thecurrent environment with the advertisement overlaid over the generatedvirtual object; and displaying the generated virtual object with theadvertisement overlaid over the generated virtual object.
 2. The methodof claim 1, wherein the user input is the search entry.
 3. The method ofclaim 1, wherein the user input is the audio input.
 4. The method ofclaim 1, further comprising: monitoring user speech for keywords; anddetecting a keyword from the user speech, wherein the selecting theadvertisement is based on the detected keyword.
 5. The method of claim1, wherein the selecting the advertisement further comprises:identifying a category of advertisement related to the user input;determining a particular respective advertisement based on a userprofile and the identified category; and selecting the particularrespective advertisement.
 6. The method of claim 1, wherein thedetermining that each respective surface of the number of surfacesdisplayed in the virtual environment is not suitable for the placementof the advertisement is based on a size of the respective surface, asviewed from the perspective of the user from which the user input wasreceived.
 7. The method of claim 1, wherein the determining that eachrespective surface of the number of surfaces displayed in the virtualenvironment is not suitable for the placement of the advertisement isbased on the lighting of the respective surface, as viewed from theperspective of the user from which the user input was received.
 8. Themethod of claim 1, wherein the determining that each respective surfaceof the number of surfaces displayed in the virtual environment is notsuitable for the placement of the advertisement is based on thecurvature of the respective surface, as viewed from the perspective ofthe user from which the user input was received.
 9. The method of claim1, wherein the virtual environment is an augmented reality environment.10. A system comprising: control circuitry configured to: receive,during presentation of a virtual environment, a user input comprising asearch entry or an audio input; select an advertisement from anadvertisement database based on the user input; determine that eachrespective surface of a number of surfaces displayed in the virtualenvironment is not suitable for placement of the advertisement based on(a) lighting of the respective surface and (b) curvature of therespective surface, as viewed from a perspective of a user from whichthe user input was received; and in response to determining that eachrespective surface of the number of surfaces displayed in the virtualenvironment is not suitable for the placement of the advertisement:determine a current environment of the virtual environment; and outputcircuitry configured to: in response to the determining via the controlcircuitry that each respective surface of the number of surfacesdisplayed in the virtual environment is not suitable for the placementof the advertisement: generate for display a virtual object related tothe current environment with the advertisement overlaid over thegenerated virtual object; and display the generated virtual object withthe advertisement overlaid over the generated virtual object.
 11. Thesystem of claim 10, wherein the user input is the search entry.
 12. Thesystem of claim 10, wherein the user input is the audio input.
 13. Thesystem of claim 10, wherein the control circuitry is further configuredto: monitor user speech for keywords; and detect a keyword from the userspeech, wherein the control circuitry is configured to select theadvertisement based on the detected keyword.
 14. The system of claim 10,wherein the control circuitry is configured to select the advertisementby: identifying a category of advertisement related to the user input;determining a particular respective advertisement based on a userprofile and the identified category; and selecting the particularrespective advertisement.
 15. The system of claim 10, wherein thecontrol circuitry is configured to determine that each respectivesurface of the number of surfaces displayed in the virtual environmentis not suitable for the placement of the advertisement based on a sizeof the respective surface, as viewed from the perspective of the userfrom which the user input was received.
 16. The system of claim 10,wherein the control circuitry is configured to determine that eachrespective surface of the number of surfaces displayed in the virtualenvironment is not suitable for the placement of the advertisement basedon the lighting of the respective surface, as viewed from theperspective of the user from which the user input was received.
 17. Thesystem of claim 10, wherein the control circuitry is configured todetermine that each respective surface of the number of surfacesdisplayed in the virtual environment is not suitable for the placementof the advertisement based on the curvature of the respective surface,as viewed from the perspective of the user from which the user input wasreceived.
 18. The system of claim 10, wherein the virtual environment isan augmented reality environment.